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ABSTRACT 

Objective: To assess the impact of a national 
immunization training certificate program on the 
perceived knowledge, skills and attitudes of 
pharmacy students toward pharmacy-based 
immunizations. 

Methods: The study design utilized a pre- and post- 
survey administered to pharmacy students before 
and after the American Pharmacists Association's 
(APhA) Pharmacy-Based Immunization Delivery 
program. The primary outcome explored was a 
change in the perceived knowledge, skills, and 
attitudes of the pharmacy students. A five-point 
Likert scale (i.e. strongly agree = 5, strongly 
disagree = 1) was used for measuring the main 
outcomes, which was summated by adding the 
individual item scores in each section to form a 
composite score for each outcome. 
Results: The certificate training program resulted in 
a significant improvement in knowledge (38.5% 
increase in score, p<0.001) and skills (34.5% 
increase in score, p<0.001), but not attitudes (1% 
increase in score, p=0.210). 
Conclusions: The national immunization training 
certificate program had a positive impact on the 
perceived knowledge and skills of pharmacy 
students. However, no change was observed 
regarding students' perceived attitudes toward 
pharmacy-based immunizations. 
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IMPACTO DE UN PROGRAMA 
CERTIFICADO DE ENSENANZA SOBRE 
CONOCIMIENTO, HABILIDADES Y 
ACTITUDES PERCIBIDOS DE LOS 
ESTUDIANTES DE FARMACIA HACIA LAS 
VACUNACIONES EN FARMACIA 

RESUMEN 

Objetivo: Evaluar el impacto de un programa 
nacional certificado de formacion sobre 
conocimiento, habilidades y actitudes percibidas de 
los estudiantes de farmacia hacia la vacunacion en 
farmacia. 

Metodos: El disefio del estudio utilizo un 
cuestionario pre- y post- administrado a los 
estudiantes de farmacia antes y despues del 
programa de la American Pharmacists Association 
de administracion de vacunas en farmacia. Los 
resultados primarios explorados fueron el cambio 
en conocimiento, habilidades y actitudes percibidos 
de los estudiantes de farmacia. Se utilizo una escala 
Likert de 5 puntos (fuertemente de acuerdo = 5, 
fuertemente en desacuerdo =1) para medir los 
resultados principales, que se totalizaba sumando la 
puntuacion de los items de cada seccion para 
obtener un valor total para cada resultado. 
Resultados: El programa de ensenanza certificado 
produjo una mejora significativa en el 
conocimiento (38,5% de mejoria en la puntuacion, 
p<0,001) y habilidades (34,5% de mejoria en la 
puntuacion, p<0,001), pero no en las actitudes (1% 
de mejoria, p=0,210). 

Conclusiones: El programa nacional certificado de 
ensenanza tuvo un impacto positivo en el 
conocimiento y habilidades percibidas de los 
estudiantes de farmacia. Sin embargo, no se 
observaron cambios en las actitudes percibidas 
havia la vacunacion en farmacia. 

Palabras clave: Vacunacion. Estudiantes de 
farmacia. Actitud del personal sanitario. Estados 
Unidos. 



INTRODUCTION 

Influenza causes an estimated 110,000 
hospitalizations and 20,000 deaths every year in the 
United States. 1 Influenza vaccination is the most 
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effective way to prevent infection from the virus; 
therefore, public health initiatives have been set 
forth to increase vaccination rates for influenza 
among other vaccinations. Allowing pharmacists to 
administer immunizations is one potential solution to 
increase vaccination rates and ultimately improve 
public health. In response to this issue, the 
American Pharmacists Association (APhA) 
developed a certificate training program for 
pharmacists, Pharmacy-Based Immunization 
Delivery, over a decade ago. 2 Importantly, the 
Center for Disease Control and Prevention's (CDC) 
National Immunization Program has officially 
recognized APhA's program. Many leaders in the 
profession agree that allowing pharmacists to 
administer immunizations has been one of the 
biggest movements towards improving direct patient 
care in recent years. 3 

With this clinical opportunity comes the need to 
determine the most appropriate method for 
delivering pharmacy-based immunization education 
to pharmacy students. The APhA certificate training 
program has been utilized at various pharmacy 
schools; however, few studies have assessed the 
impact of such a program on pharmacy students. 4 ' 5 
A recent article by Welch et al. describes one 
approach to incorporating an immunization 
certificate training program into a pharmacy 
curriculum. The program was delivered over 3 
weeks in the spring semester of the third 
professional year (P3) as part of a required 
Pharmacy Care Lab sequence. The study found 
that more than 44% of students provided positive 
feedback regarding the course with about 10% 
reporting that the course should not be mandated in 
the curriculum. 5 This study is limited by the fact that 
it was completely qualitative in nature and did not 
extensively evaluate the program. Given this 
background, the objective of our study was to 
provide a comprehensive evaluation of an 
immunization training certificate program by 
determining its impact on the perceived knowledge, 
skills and attitudes of pharmacy students toward 
pharmacy-based immunizations. 

METHODS 

As a brief background, Butler University College of 
Pharmacy & Health Sciences pharmacy students 
complete 4 years of professional pharmacy 
academic training, with the final year (P4) 
comprised mostly of advanced pharmacy practice 
experiences (APPEs). However, no APPE rotations 
are scheduled in the month of December at which 
time the pharmacy students may enroll in the APhA 
Pharmacy-Based Immunization Delivery program. 
The immunization program consists of 20 hours of 
continuing education credit obtained via self-study 
(12 hours) and a live training session (8 hours). 2 
The live course seminar for this study was held at 
Butler University over 2 days in December 2007 and 
was taught by two APhA certified immunization 
trainers. 

A self-administered, pre- and post- electronic survey 
was developed to meet the study objective. The 
pre-survey was distributed through e-mail to the 



students enrolled in the immunization elective prior 
to the live course segment and prior to receiving the 
self-study materials. Based on the results of the pre- 
survey, collaborative practice agreements was 
determined to be an unfamiliar topic for most 
students. Therefore, an additional lecture on 
collaborative practice agreements was developed 
and presented by the author during the live course 
seminar. After the live training session, the 
participants were provided the post-survey, which 
consisted of the same questions as the pre-survey 
as well as three additional questions assessing the 
overall program. Participation in the immunization 
elective course and this study was optional for all 
current P4 pharmacy students at Butler University, 
which included 160 students at the time of this 
study. The student certificate fee was paid by a 
University grant. 

Survey Development and Completion 

An electronic survey software program (Survey 
Monkey©) was used to develop and administer the 
survey. The pre-survey consisted of a cover letter 
and four sections: demographics, knowledge, skills 
and attitudes. The demographics section included 
age, gender, and previous injection experience, 
including prior subcutaneous and intramuscular 
injection instruction and/or hands-on experience. 
The knowledge section consisted of six items that 
assessed the student's knowledge relating to 
immunizations. The skills section consisted of six 
items that assessed the skills of the student related 
to immunizations. The attitudes section consisted of 
six items that assessed the attitudes of the student 
toward pharmacy-based immunizations. Three 
additional questions were included with the post- 
survey to assess the student's level of confidence in 
their ability to provide immunizations, to implement 
an immunization program, and an open-ended 
program feedback question. The participants were 
asked to indicate their level of agreement with the 
survey items in each section using a 5-point Likert 
scale from 1 (strongly disagree) to 5 (strongly 
agree). A composite score for each survey section 
was derived by awarding points for each survey 
item reply and summing the total points for each 
section. Points were assigned as follows: 5 points 
(strongly agree), 4 points (agree), 3 points (neutral), 
2 points (disagree), and 1 point (strongly disagree). 
Therefore, the maximum composite score for each 
section was 30 points (i.e., 5 points — strongly 
agree — for all 6 items = 30 points), and the 
minimum composite score for each section was 5 
points (i.e., 1 point — strongly disagree — for all 6 
items = 5 points). 

Outcomes 

The primary outcome was a change in the 
pharmacy students' perceived knowledge, skills and 
attitudes toward pharmacy-based immunizations. 
Secondary outcomes included the perceived level of 
confidence the pharmacy students had toward 
providing immunizations and implementing 
immunization programs. 
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Table 1: Baseline Demographic Data (n=57) 


Gender - n (%1 
Male 
Female 


15 (26.3%) 
42 (73.7%) 


Mean (SD) 


23.8 (SD=1.7) 


1 have been instructed how to administer a subcutaneous injection to a patient. - n (%) 

Yes 

No 


51 (89.5%) 
6(10.5%) 


1 have been instructed how to administer an intramuscular injection to a patient. - n (%) 

Yes 

No 


19(33.3%) 
38 (66.7%) 


1 have administered a subcutaneous injection to a patient. - n (%) 

Yes 

No 


16 (28.1%) 
41 (71.9%) 


1 have administered an intramuscular injection to a patient, n - (%) 

Yes 

No 


4 (7.0%) 
53 (93.0%) 



Statistical Analysis 

Descriptive statistics were performed on the 
demographic data. Since data provided by Likert 
scales is ordinal and tends to be non-normally 
distributed, non-parametric tests should ideally be 
conducted for evaluation. However, all statistical 
findings were the same for both the parametric (t- 
test) and non-parametric (Mann-Whitney U test) 
tests; therefore, the results are reported based on 
parametric t-tests. All statistical analyses were 
performed using Statistical Package for Social 
Sciences version 13.0 (SPSS Inc., Chicago, IL). 
The study was approved by the Butler University 
Institutional Review Board. 

RESULTS 

Response Rate 

A total of 57 students enrolled in the Immunization 
Elective out of 160 total students in the P4 class. 
The response rate for the pre-survey was 100% 
(57/57). The response rate for the post-survey was 
94.7% (54/57). 

Baseline Characteristics and Additional Post-survey 
Questions 

The majority of participants in the Immunization 
Elective were female (Table 1). The mean age of 
the participants was 23.8 (SD=1.7) years. Overall, 
89.5% of the participants had been instructed how 
to administer a subcutaneous injection to a patient 
while only 28.1% had actually administered a 
subcutaneous injection. Prior to the immunization 
training program, 33.3% of the participants had 
been instructed how to administer an intramuscular 
injection to a patient while only 7.0% had actually 
administered an intramuscular injection. This prior 
experience was obtained primarily through a 
Pharmacy Care Lab earlier in the curriculum in 
which students voluntarily practiced subcutaneous 
injection technique during the diabetes care lab. 

After completing the immunization certificate 
program, more than 98% of the participants felt 
confident (strongly agree or agree) in providing 
immunizations to patients (Table 2). Furthermore, 
about 85% of the participants felt confident (strongly 



agree or agree) in implementing an immunization 
program in their practice setting after their training. 
Most participants felt that the hands-on practice with 
the injection techniques and learning the CDC 
immunization schedules were the two most 
beneficial components of the certificate training 
program. 



Table 2: Additional Post-survey Questions (n=54) 



Upon completion of the immunization certificate 
program, I am confident about providing immunizations 
to patients. - % (n) 



Strongly Agree 
Agree 
Neutral 
Disagree 

Strongly Disagree 



63.0% (34) 
35.2% (19) 
1.9% (1) 
0% (0) 
0% (0) 



Upon completion of the immunization certificate 
program, I am confident about implementing an 
immunization program in my practice setting. - n (%) 



Strongly Agree 
Agree 
Neutral 
Disagree 

Strongly Disagree 



27.8% (15) 
57.4% (31) 
9.3% (5) 
5.6% (3) 
0% (0) 



Provide at least two things that were the most 
beneficial about completing the immunization 
certificate program. 

Most common responses: 

■ Hands-on practice with injection techniques 
(subcutaneous/intramuscular) 
■ Learning the CDC immunization schedules 



Knowledge 

The primary outcome results can be seen in Table 
3. The overall mean composite score in the 
knowledge section increased by 38.5% between the 
pre- and post-survey (18.47 SD=3.29 vs. 25.59 
SD=2.58, respectively; p<0.001). All the items in the 
knowledge section showed statistically significant 
differences between pre- and post- scores (p<0.001 
for all items; Table 4). 



Table 3: Main Outcomes - Knowledge, Skills, and 
Attitudes 


Outcome 


Pre-Survey 
Mean (SD) 
(n=57) 


Post-Survey 
Mean (SD) 
(n=54) 


P-value 


Knowledge 


18.47 (3.29) 


25.59 (2.58) 


<0.001 


Skills 


19.33 (3.87) 


26.01 (2.72) 


<0.001 


Attitudes 


26.91 (2.01) 


27.42 (2.27) 


0.210 
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Skills 

The overall mean composite score in the skills 
section increased by 34.5% between the pre- and 
post-survey (19.33 SD=3.87 vs. 26.01 SD=2.72, 
respectively; p<0.001). All the items in the skills 
section showed statistically significant differences 
between pre- and post- scores (p<0.001 for all 
items; Table 4). 

Attitudes 

The overall mean composite score in the attitudes 
section increased by 1% between the pre- and post- 
survey (26.91 SD=2.01 vs. 27.42 SD=2.27; 
p=0.210). The only item in the attitudes section that 
showed a statistically significant difference between 
pre- and post- scores was the following statement: 
The APhA immunization training program will 
prepare me to become an immunizer for the 
community (p=0.035; Table 4). 



training certificate program had a significant positive 
impact on the knowledge and skills of P4 pharmacy 
students. However, its impact on their attitudes was 
not significant. One possible explanation for this 
finding is that the students who enrolled in the 
immunization elective course were those students 
who already had positive attitudes toward 
immunizations. In fact, the overall mean score for 
the attitudes section was the highest of all three 
sections at baseline. Therefore, there was less 
room for improvement in this section because the 
students who enrolled in the class were those 
students who had positive attitudes toward 
pharmacy-based immunizations, representing a 
ceiling effect. Moreover, it is useful to see that the 
APhA training program had a positive impact on the 
knowledge and skills of students enrolled in the 
program. 

Establishing collaborative practice agreements was 
an unfamiliar area for most students prior to the 
training program. In order to address this issue, a 



Table 4: Individual Items for Main Outcome Sections 


Item 


Mean Pre-survey 
Score + SD (n=57) 


Mean Post-survey 
Score + SD (n=54) 


P-value 






I am knowledgeable on the current CDC guidelines for adult 
immunizations. 


3.21 +0.881 


4.35 + 0.520 


< 0.001 


I am knowledgeable on the mechanisms of action and 
contraindications of vaccines. 


3.23 + 0.824 


4.24 + 0.612 


< 0.001 


I am knowledgeable on vaccine-preventable disease. 


3.60 + 0.863 


4.43 + 0.536 


< 0.001 


I am knowledgeable on biohazard waste disposal and blood 
born pathogen regulations. 


3.91+0.931 


4.48 + 0.540 


< 0.001 


I know how to obtain reimbursement for providing adult 
immunizations. 


1.77 + 0.708 


3.65 + 0.805 


< 0.001 


I know how to manage a vaccine adverse reaction. 


2.75 + 1.005 


4.44 + 0.538 


< 0.001 


Skills 


I know how to communicate effectively with other health 
care professionals regarding immunization schedules. 


3.56 + 0.824 


4.41+0.533 


< 0.001 


I know how to communicate effectively with patients 
regarding immunization schedules and recommendations. 


3.61+0.796 


4.43 + 0.536 


< 0.001 


I know how to obtain informed consent from a patient. 


3.91+0.830 


4.50 + 0.541 


< 0.001 


I know how to prioritize patients based on their risk for 
contracting a vaccine-preventable disease (i.e. high-risk 
group vs. low-risk group) 


3.53 + 0.889 


4.57 + 0.536 


< 0.001 


I know how to create and implement immunization protocols. 


2.39 + 0.901 


4.11 +0.604 


< 0.001 


I know how to establish collaborative practice agreements 
with community physicians. 


2.33 + 0.913 


4.00 + 0.727 


< 0.001 


Attitudes 


Pharmacists are capable of safely and effectively 
administering immunizations with proper training. 


4.77 + 0.535 


4.91 +0.351 


0.116 


I will look for future jobs that will allow me to administer 
immunizations. 


4.12 + 0.683 


4.20 + 0.711 


0.542 


The APhA immunization training program will prepare me to 
become an immunizer for the community. 


4.56 + 0.501 


4.76 + 0.473 


0.035 


Pharmacy students are capable of administering vaccines to 
patients with proper training. 


4.30 + 0.680 


4.39 + 0.656 


0.477 


Allowing pharmacists to administer vaccines will improve job 
satisfaction. 


4.40 + 0.623 


4.33 + 0.727 


0.585 


Allowing pharmacists to administer vaccines will improve 
public health. 


4.75 + 0.474 


4.83 + 0.423 


0.357 



DISCUSSION 

Our study appears to be some of the only published 
evidence to date on the impact of an immunization 
training program on the perceived knowledge, skills 
and attitudes of pharmacy students toward 
pharmacy-based immunizations. The results of this 
study indicate that the national immunization 



20-minute lecture on collaborative practice 
agreements was developed and presented during 
the live training, which made a significant difference 
on this skill item. Specifically, the mean pre-survey 
score for this item was 2.33 (SD=0.913), and the 
mean post-survey score was 4.00 (SD=0.727) 
(p<0.001). The APhA certificate training program 
has been developed in such a way that allows for 
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tailoring of the program to the individual audience's 
needs. This individualized approach to teaching the 
certificate training program should remain in future 
sessions to help provide individualized and 
optimized education. 

In looking at the recent study by Welch et al., they 
recommend incorporating immunization training into 
curricula based on the professional needs of the 
students, student scheduling, and faculty workload. 5 
They found success in placing the required course 
in the P3 year as part of a Pharmacy Care Lab 
while our study was an elective course offered 
during winter break of the P4 year. Since almost 
every participant felt confident in providing 
immunizations to patients after completing the 
training, the P4 year may be a logical time for 
immunization training due to its proximity to 
graduation. Under some state's current legislation 
(including Indiana), pharmacy students are not 
allowed to provide immunizations; therefore, 
providing this education earlier in the curriculum 
would not be optimal in such states. Additionally, 
the temporal effects of retention of knowledge could 
potentially diminish such high confidence levels 
among the students. The elective nature of the 
course is also debatable. The immunization 
certification being optional is advantageous to 
students who know they will not be in a pharmacist 
position that allows them to administer 
immunizations and can, therefore, opt out of taking 
the course. However, making the course required 
would increase the number of potential immunizing 
pharmacists, which would ultimately help improve 
public health. Also, offering the training over a 2-day 
period allowed students a focused and intensive 
period of time to complete the training. In this way, 
the program did not interfere with any of the 
semester curricular workload. Other methods of 
implementation should certainly be evaluated and 
assessed at other Colleges/Schools of Pharmacy. 

In order to help achieve meet the American 
Association of Colleges of Pharmacy Center for the 
Advancement of Pharmaceutical Education (CAPE) 
2004 Educational Outcomes regarding public health 
and to meet the current and future demands of 
patients, immunization education has a clear place 
in the pharmacy curriculum. 6 Therefore, allocating 
appropriate funding should be considered a top 
priority for pharmacy schools to help meet this 
objective and ultimately improve public health. 
Finally, at a time when an influenza pandemic is a 
major public health concern, it is promising to know 
that pharmacists can play an active role in helping 
combat this potential crisis and other future crises. 
Increasing the number of certified pharmacist 
immunizers will certainly place pharmacists in a 
position to aid in such efforts. 

There are limitations to this study which deserve 
mention. First, because the study was a self- 
administered survey, the data were self-reported. It 
should be noted that previous research has shown 
a mismatch between pharmacy student's ability to 
self-assess as compared to external assessors (i.e., 
pharmacists) in the following global rating scale 
areas of patient interaction: verbal communication, 



nonverbal communication, degree of focus, logic, 
coherence in the interview, empathy, and overall 
performance. The differences were the largest in 
the areas of empathy, and logic/focus/coherence of 
interviewing. 7 Conversely, research has shown that 
pharmacy student self-assessments were similar to 
faculty ratings in the context of a pre- and post- 
survey for a drug information APPE. 8 Because our 
study relied on student self-assessment, there is the 
potential for inaccuracy in their actual knowledge 
and skills relating to immunizations. However, all of 
the students passed the final exam and, thus, 
passed the course. Regardless, pharmacy student 
self-assessment should be an area of future 
research as it is a critical skill for career and 
personal development. Also, although the survey 
instrument was constructed with the assistance of a 
pharmacy practice and a social and administrative 
pharmacy professor, this measurement instrument 
did not undergo rigorous reliability and validity 
testing prior to being used in this study. Therefore, 
the items used to assess the knowledge, skills and 
attitudes may not have truly captured the entire 
universe of the construct that was being measured. 
Furthermore, a common limitation with survey 
instruments is the potential for misinterpretation of 
survey items. Specifically, items in the knowledge 
and skills sections may have some overlap; 
however, particular attention was paid to this issue 
in the development of the survey to minimize any 
potential confusion. A potential threat to internal 
validity was that the lecture on collaborative practice 
agreements was developed and presented as a 
result of the low mean score on the pre-test. By 
directly incorporating this component into the 
training program, the internal validity of the study 
was threatened; but, this potential threat is difficult 
to avoid when using an intervention such as a 
training program. Since approximately 36% (i.e., 57 
of 160) of the P4 class enrolled in the elective 
program, the potential for selection bias does exist 
in that those students already knowledgeable on 
immunizations may have been less likely to enroll in 
the course, leading to a more impactful group of 
students enrolling in the course. Finally, it is not 
known whether the results of this study are 
generalizable to other Colleges/Schools of 
Pharmacy. 

CONCLUSIONS 

A positive impact on the perceived knowledge and 
skills of P4 professional pharmacy students was 
observed with the implementation of a national 
immunization training certificate program. However, 
the perceived attitudes of the pharmacy students 
did not significantly change in this study. Increasing 
the number of certified immunizing pharmacists is 
essential to meet the current and future demand for 
increasing vaccination rates. As the scope of 
pharmacy practice continues to expand and more 
legislation is approved for pharmacists to provide 
immunizations, ongoing assessment is necessary to 
identify the most effective instructional methods for 
the delivery of immunization education for pharmacy 
students. 
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